Robot and method for controlling the same

ABSTRACT

A robot includes: a main body; a traveling mechanism attached to the main body to move the main body; an image pickup device which picks up an image of an object in a space including a traveling plane; an object detector which detects the object existing in the space based on an image picked up by the image pickup device; a distance calculator which calculates a distance to the object detected by the object detector; a degree of caution setting device which sets a degree of caution with respect to the object detected by the object detector based on the image picked up by the image pickup device and the distance calculated by the distance calculator; and a controller which controls the traveling mechanism based on the degree of caution set by the degree of caution setting device and the distance calculated by the distance calculator.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority fromprior Japanese Patent Application No. 2007-36612 filed on Feb. 16, 2007in Japan, the entire contents of which are incorporated herein byreference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a robot and its control method.

2. Related Art

In the conventional art, the robot searches for information concerningan obstacle around the robot by using an outside world sensor such as anultrasonic wave sensor or a range finder. If an unexpected obstacle isdetected, the robot operates to avoid the obstacle while decelerating(see, for example, JP-A 2005-310043 (KOKAI)). If the obstacle proves tobe a traveling object, the robot assumes an action strategy ofcontinuing to stop until the traveling object disappears from thevicinity of the robot (see, for example, JP-A 3-282711 (KOKAI)).

The robot observes a time change value of the information obtained fromthe outside world sensor in order to determine whether the obstacle is atraveling object. By in addition conducting image processing such astemplate matching or optical flow using a camera, the robot makes adecision whether the obstacle is a traveling object or makes a decisionwhether the obstacle is a human being in some cases. However, moredetailed sorting and analysis are hardly conducted.

An obstacle discrimination technique utilizing height information of theobstacle is also known (see, for example, JP-A 2004-326264 (KOKAI)).According to the technique described in JP-A 2004-326264 (KOKAI),however, the height information is used to detect a stationaryenvironmental obstacle, such as a wall or a pillar, or a human being. InJP-A 2004-326264 (KOKAI), a technique for detecting the posture state ofa human being is mentioned, but discrimination of the human beingincluding the degree of caution is not taken into consideration.Furthermore, JP-A 2004-326264 (KOKAI) discloses an obstacle detectionand discrimination technique, and a concrete operation change techniqueagainst the obstacle is not presented.

Introduction of autonomous traveling objects such as robots into storesis under study. Considering the introduction of robots into generalenvironments such as stores or facilities, however, it becomes importantto ensure safety of the robots. Especially in the environment in thestate of coexistence with human beings, it is necessary to take safetyoperation for human being's recognition and human beings who performrecognition. On the other hand, it is also necessary to ensure the basicperformance, such as smooth traveling operation, of the robots, andtrade off between the safety and the traveling performance occurs.

When a robot has detected a human being, the robot stops on the spot andwaits for the human being to go away from the vicinity of the robot, orthe robot avoids the with a sufficient spacing between, under presentconditions. In a narrow place such as a store, however, it is difficultto ensure a sufficient spacing and consequently the robot cannot helpstopping and letting the human being go past. In such an operationstrategy, however, the work efficiency of the robot is poor and therobot cannot withstand practical use.

As one of reasons why only such a control technique can be used, it canbe mentioned that information concerning other traveling objects therobot can know is little and an action control law which can be appliedto all detected persons is used.

SUMMARY OF THE INVENTION

The present invention has been made in view of these circumstances, andan object thereof is to provide a robot which performs safe operationand has a high business capability in stores, and a control method forsuch a robot.

According to an aspect of the present invention, there is provides arobot including: a main body; a traveling mechanism configured to beattached to the main body and to move the main body; an image pickupdevice configured to pick up an image of an object in a space includinga traveling plane; an object detector configured to detect the objectexisting in the space based on an image picked up by the image pickupdevice; a distance calculator configured to calculate a distance to theobject detected by the object detector; a degree of caution settingdevice configured to set a degree of caution with respect to the objectdetected by the object detector based on the image picked up by theimage pickup device and the distance calculated by the distancecalculator; and a controller configured to control the travelingmechanism based on the degree of caution set by the degree of cautionsetting device and the distance calculated by the distance calculator.

According to an another aspect of the present invention, there isprovided a control method for a robot including a main body, a travelingmechanism configured to be attached to the main body and to move themain body, an image pickup device configured to pick up an image of anobject in a space including a traveling plane, an object detectorconfigured to detect the object existing in the space based on an imagepicked up by the image pickup device, a distance calculator configuredto calculate a distance to the object detected by the object detector,and a controller configured to control the traveling mechanism,

the control method including: setting a degree of caution with respectto the object detected by the object detector based on the image pickedup by the image pickup device and the distance calculated by thedistance calculator; and causing the controller to control the travelingmechanism based on the degree of caution and the distance calculated bythe distance calculator.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view of a robot according to an embodiment ofthe present invention;

FIG. 2 is a flow chart showing operation of a robot according to anembodiment;

FIG. 3 is a flow chart showing a decision operation conducted in a robotaccording to an embodiment to determine whether an object is anobstacle;

FIG. 4 is a flow chart showing a decision operation conducted in a robotaccording to an embodiment to determine whether an obstacle is atraveling object;

FIG. 5 is a flow chart showing a decision operation conducted in a robotaccording to an embodiment to determine whether an obstacle is a humanbeing;

FIG. 6 is a flow chart showing a decision operation conducted in a robotto determine whether a detected human being is a human being of a highcaution level;

FIG. 7 is a schematic diagram showing a detected traveling object regionin an image picked up by a camera;

FIG. 8 is a flow chart for explaining a first human being's heightestimation technique;

FIG. 9 is a diagram for explaining a table showing correspondencebetween an image region height P|y| according to a distance d(φ) to atraveling object and a real height to the traveling object;

FIG. 10 is a flow chart for explaining a second human being's heightestimation technique;

FIG. 11 is a diagram for explaining a second human being's heightestimation technique; and

FIG. 12 is a diagram showing a traveling restriction law used by a robotaccording to an embodiment.

DESCRIPTION OF THE EMBODIMENTS

A robot according to an embodiment of the present invention is shown inFIG. 1. A robot 1 according to the present embodiment is a robot ofindependent two-wheel drive type having drive wheels 10 on the right andthe left and free casters (not illustrated) in front and in the rear. Asa result, the robot 1 can move in the space freely. In other words, onlythe left wheel or the only the right wheel can be rotated around a leftwheel axle and a right wheel axle, respectively.

A monitor, an input keyboard and a mouse can be connected to the robot1. As a result, it becomes possible to present internal information ofthe robot 1 to the outside and transmit external intention to the robot1. The robot 1 has a function of displaying a virtual space divided intounit grids on the monitor. The robot 1 has a function of allowing amanager to form a region in the virtual space by successively linkinggrid intersections with the mouse. The robot 1 has a function ofallowing the manager to generate some spaces and divide and couple thespaces. Furthermore, the robot 1 has a function of causing the robot 1to recognize a traveling space structure of itself by specifying whetherthe region is a region where the robot 1 can travel.

The robot 1 has a laser range finder 12. Topography around the robot 1can be measured accurately by using the laser range finder 12.Ultrasonic wave sensors 14 are installed on the circumference of thetrunk of the robot 1. A distance to an obstacle in the sensorinstallation direction can be measured by an ultrasonic wave sensor 14as well. The robot 1 further has two cameras 16 a. A distance to theobstacle in a direction viewed from the cameras 16 a can also becalculated by calculating the distance in the depth direction of theimage with stereo viewing using the cameras 16 a. The depth distance inthe camera angle of view may also be measured by using a distance imagesensor which is not illustrated.

Furthermore, the number of revolutions of the wheels 10 is measured byan encoder. The location of the robot 1 is estimated by accumulating thenumber of revolutions and successively calculating the path of thewheels 10. The robot 1 has a radio signal sender (not illustrated)besides. A plurality of antennas installed in the space receive a radiowave signal with an identification ID emitted from the robot 1. Anenvironment system installed in the space estimates a radio wave sendingsource on the basis of received strength distribution at this time. Anestimation device regards the location of the radio wave sending sourceas the location of the robot 1 itself. In addition, a plurality of radiowave senders in each of which the installation location information isalready known and the sending time and code of the sending signal can bediscriminated are installed in a store. The robot 1 has an antenna (notillustrated) which can receive signals from these senders. The robot 1can also estimate the location of itself on the basis of information ofthese kinds as in the GPS (Global Position System).

The robot 1 can also recognize a traveling space structure of itself byautonomously wandering about in an unknown space, detecting thesurrounding topography on the basis of a relative location from a basepoint and a distance from an obstacle in the surroundings estimated froman input of an outside world sensor at that time, and traveling andsearching in the whole region in the traveling space.

The robot 1 has a camera unit 16 in its head. As for the camera unit 16,the camera posture can be controlled in the pan direction and the tiltdirection independently of the trunk movement.

The robot 1 generates a traveling route along which the robot 1 canreach a destination without colliding with a wall or a pillar, on thebasis of map information.

The robot 1 has a touch panel 18 on its trunk. An external intention canalso be input via the touch panel 18. The robot 1 has a plurality ofmicrophones 20 on its trunk. An external intention can also be input tothe robot 1 by voice via the microphones 20.

In the present embodiment, the robot 1 is supposed to travel in a store.

Upon detecting an obstacle, the robot 1 changes its operation accordingto the kind of the obstacle. Operation decision flows at this timebecome as shown in FIGS. 2 to 9. The robot 1 makes a decision whetherthere is an obstacle by using the outside world sensor such as the laserrange finder 12 (step S1 in FIG. 2). The decision whether there is anobstacle is made, for example, as shown in FIG. 3.

First, a distance d(φ) to an object in a detection direction φ of theoutside world sensor is measured (step S11 in FIG. 3). The distance d(φ)is compared with an obstacle decision threshold D (step S12 in FIG. 3).If the distance d(φ) is equal to or greater than the obstacle decisionthreshold D, then the object is not judged to be an obstacle (step S13in FIG. 3), and an approach permission distance L_(max) between therobot 1 and an obstacle is set to L_(max)=0 without placing a limitationon L_(max) (step S2 in FIG. 2).

If the distance d(φ) is less than the obstacle decision threshold D,then the robot 1 proceeds to step S14 shown in FIG. 3. At the step S14,a distance L(φ) to a wall or a pillar in the detection direction φ andan object marked on a map is calculated on the basis of map data therobot 1 has and the current location and posture of the robot 1, andd(φ) is compared with L(φ). If the distance d(φ) substantially coincideswith L(φ), the detected object is judged to be a topography obstacle(step S15 in FIG. 3). If the distance d(φ) is longer than the distanceL(φ), then the robot 1 judges its own location to be recognized falselyand conducts re-identification of its own location (step S15 in FIG. 3).On the other hand, if the distance d(φ) is shorter than the distanceL(φ), then the robot 1 recognizes the object as an unexpected obstaclewhich is not on the map (step S16 in FIG. 3). The decision whether thereis an obstacle is thus finished.

If the robot 1 recognizes the obstacle as an unexpected obstacle, thenthe robot 1 proceeds to step S3 shown in FIG. 2 to make a decisionwhether the obstacle is a traveling object. The decision whether theobstacle is a traveling object is made, for example, as shown in FIG. 4.

First, a search is conducted in that direction φ by using the outsideworld sensor again (step S31 in FIG. 4). A decision is made whetherthere is a change in distance or direction to the obstacle with timeelapse (step S32 in FIG. 4). For example, a difference between anorientation φ(t1) of the obstacle at time t1 and an orientation φ(t2) ofthe obstacle at time t2 preceding the time t1, i.e. an orientationchange is represented as φ(t1)−φ(t2). And a difference between adistance d(t1) to the obstacle and a distance d(t2), i.e., a distancechange is represented as d(t1)−d(t2). A decision whether the differenceφ(t1)−φ(t2) and the difference d(t1)−d(t2) are respectively greater thandecision thresholds Dφ and Dd is made at step S32. If the differenceφ(t1)−φ(t2) and the difference d(t1)−d(t2) are respectively greater thanthe decision thresholds Dφ and Dd, then the robot 1 proceeds to step S33and the obstacle is judged to be a traveling object. If the robot 1itself travels at this time, the orientation change and the distancechange are corrected according to a change in its own location.

On the other hand, if the orientation change and the distance change arerespectively equal to or less than the decision thresholds Dφ and Dd,then the robot 1 turns the cameras to the direction, acquires an image,measures an optical flow from the image picked up by the cameras, andmakes a decision whether a flow vector different from the background isdetected in the obstacle detection direction (step S34 in FIG. 4). If aflow vector different from the background is detected, then the robot 1proceeds to step S33 shown in FIG. 4 and the obstacle is judged to be atraveling object. If the flow vector is not detected, then the robot 1proceeds to step S35 and the obstacle is judged not to be a travelingobject. In this case, the robot 1 proceeds to step S4 shown in FIG. 2,sets the level of the approach permission distance limitation (degree ofcaution) of the robot 1 to the obstacle to Lv. 1, and sets the approachpermission distance L_(max) to L_(max)=L₁ (>0). The decision whether theobstacle is a traveling object is thus finished.

If the robot 1 recognizes the obstacle as a traveling object, then therobot 1 proceeds to step S5 shown in FIG. 2 and the robot 1 makes adecision whether the obstacle is a human being. This decision is made,for example, as shown in FIG. 5.

First, the robot 1 makes a decision whether an image distance is alreadyknown on the basis of an image picked up by the cameras in the φdirection (steps S51 and S52 in FIG. 5). If the image distance isalready known, then the robot 1 proceeds to step S53 shown in FIG. 5,cuts out only an image near the distance d(φ) to the obstacle, and thenproceeds to step S54 shown in FIG. 5. If the image distance is notalready known (for example, if the camera is a stereoscopic camera),then the robot 1 proceeds to step S54 shown in FIG. 5.

In the step S54 shown in FIG. 5, the robot 1 conducts image processingand calculates a human being's shape correlation value P. In otherwords, if the robot 1 has a distance image sensor, the robot 1 detects acontour shape. If the robot 1 detects a traveling object from the flowvector, then the robot 1 conducts contour extraction processing on itsdetected peripheral region, conducts template matching with a humanbeing shape pattern retained within the robot 1 as already knowninformation, and calculates the human being's shape correlation value P.Subsequently, the robot 1 proceeds to step S55 shown in FIG. 5, andcompares the human being's shape correlation value P with a thresholdDp. If the human being's shape correlation value P is equal to orgreater than the threshold Dp, then the robot 1 proceeds to step S56shown in FIG. 5 and judges the obstacle to be a human being. On theother hand, if the human being's shape correlation value P is less thanthe threshold Dp, then the robot 1 proceeds to step S57 shown in FIG. 5and judges the obstacle not to be a human being. In this case, the robot1 proceeds to step S6 shown in FIG. 2, sets the level of the approachpermission distance limitation (degree of caution) of the robot 1 to theobstacle to Lv. 2, and sets the approach permission distance L_(max) toL_(max)=L₂ (>L₁). The human being's shape correlation value P can beobtained by matching pixel color information in a peripheral region fromwhich the flow vector has been detected with color information of a skincolor series. Even if the human being's shape correlation value P is notobtained, the traveling object may be judged to a human being in thecase where feature points such as eyes, a nose and a mouth and theirgeometrical arrangements are confirmed near a head in a traveling objectimage region. The decision whether the obstacle is a human being is thusfinished.

Subsequently, the robot 1 proceeds to step S7 shown in FIG. 2, and makesa decision whether the traveling object recognized as a human being is ahuman being of high caution level, i.e., a human being which is higherin caution level than a general human being. Since the robot 1 in thepresent embodiment has means for detecting information of the distanceto the traveling object, the actual size of the traveling object can beestimated on the basis of the information of the distance to thetraveling object and the image size of the traveling object. Thedecision whether the traveling object is a human being of high cautionlevel is made, for example, as shown in FIG. 6. The decision operationshown in FIG. 6 is made supposing that the human being of high cautionlevel is a child.

First, the robot 1 proceeds to step S71 shown in FIG. 6 and makes adecision whether an image in a longitudinal direction of a human beingpicked up is accommodated in a picked-up image region on the basis ofthe image picked up by the cameras. This decision is made as describedhereafter. The robot 1 finds a top longitudinal direction location PHyand a bottom longitudinal direction location PLy in an image region ofthe traveling object and a top location Fy_max and a bottom locationFy_min of the camera image shown in FIG. 7, and makes a decision whetherPHy<Fy_max and PLy>Fy_min (step S71 in FIG. 6). If the image in thelongitudinal direction of the human being picked up is accommodated inthe picked-up image region, then the robot 1 proceeds to step S72 andestimates the height of the human being picked up, by using a firstheight estimation technique which will be described later. If the imagein the longitudinal direction of the human being picked up is notaccommodated in the picked-up image region, then the robot 1 proceeds tostep S73 shown in FIG. 6 and makes a decision whether a head of thehuman being picked up is accommodated in the picked-up image region.This decision is made by determining whether the relation PHy<Fy_max issatisfied. If the relation PHy<Fy_max is satisfied, then the robot 1proceeds to step S74 shown in FIG. 6 and estimates the height of thehuman being picked up, by using a second height estimation techniquewhich will be described later. If the relation PHy<Fy_max is notsatisfied, then the robot 1 drives tilt axes of the cameras and movesthe cameras upward in the tilt direction so as to prevent the travelingobject image region from protruding from the visual field of the camerasas regards at least the upward direction (Step S75. Thereafter, therobot 1 picks up the image of the traveling object (human being) again,takes in the picked-up image, returns to the step S71 shown in FIG. 6,and repeats the above-described operation.

The above-described first height estimation technique will now bedescribed with reference to FIGS. 8 and 9. The robot 1 in the presentembodiment has a table showing correspondence between an image regionheight P|y| according to the distance d(φ) to the traveling object and areal height to the traveling object. In the first height estimationtechnique, therefore, the robot 1 measures the bottom location PLy andthe top location PHy shown in FIG. 7 as regards the vertical directionof the screen of the picked-up image region of the detected person, andestimates the height of the detected person on the basis of acorrespondence table according to a distance between objects (=PHy−PLy)at that time. The estimation of the height is conducted as describedhereafter. First, the height h of the person is estimated on the basisof the extracted person's image and the distance d(φ) to the person(steps S711 and S712 in FIG. 8). This estimation is conducted on thebasis of a table showing correspondence between an image region heightP|y| and a real height h of the actual obstacle (person) as shown inFIG. 9. The estimated height h is compared with a threshold Dh (forexample, an average height value of the lowest grades of elementaryschools (for example, ten-year-old children)) at step S713 shown in FIG.8. If the height h is greater than the threshold Dh, then the robot 1proceeds to step S714 shown in FIG. 8 and judges that the extractedperson is not a person of caution level. In this case, the robot 1further proceeds to step S8 shown in FIG. 2, sets the level of theapproach permission distance limitation (degree of caution) of the robot1 to the person to Lv. 3, and sets the approach permission distanceL_(max) to L_(max)=L₃ (>L₂).

On the other hand, if the height h is equal to or less than thethreshold Dh, then the robot 1 proceeds to step S715 shown in FIG. 8 andjudges that the extracted person is a person of high caution level,i.e., a person who is higher in caution level than general persons. Inthis case, the robot 1 further proceeds to step S9 shown in FIG. 2, setsthe level of the approach permission distance limitation (degree ofcaution) of the robot 1 to the person to Lv. 4, and sets the approachpermission distance L_(max) to L_(max)=L₄ (>L₃).

By the way, in the first height estimation technique, a linear ornon-linear correspondence function may be prepared instead of theabove-described correspondence table. Strictly speaking, the firstheight estimation technique can be used only when the whole of thetraveling object is accommodated in the pick-up screen of the cameras.It is permissible that the traveling object slightly gets out of thepick-up screen as regards the lateral direction of the traveling object.As regards the vertical direction of the traveling object, however, itis necessary that the traveling object is accommodated in the screen.

The above-described second height estimation technique will now bedescribed with reference to FIGS. 10 and 11. The second heightestimation technique is an estimation technique to be used when thebottom point of the image region of the detected traveling objectreaches the bottom side of the image picked up by the cameras, i.e.,when the traveling object protrudes from the visual field of the camerasin the downward direction of the screen. According to the second heightestimation technique, the robot 1 finds a relative elevation angle or arelative depression angle α of the top point “a” of the pick-up imageregion of the traveling object from the optical axis of the camerasshown in FIG. 11, finds a relative posture relation (here, an angle β inthe tilt direction) between a camera pan tilt head and a robot main bodyon the basis of an inner world sensor such as an encoder, estimates anactual height h of the point “a” on the basis of installation states ofthe robot main body, the camera pan tilt head and the outside worldsensor previously stored as already known information, the distanceinformation d(φ) obtained by using the outside world sensor, and thegeometric relations (steps S721 and S722 in FIG. 10), and regards theactual height h of the point “a” as the person's height. In the presentembodiment, a traveling object in a store is considered. Therefore,every person detected as a traveling object is supposed to be standing.In the case of a stereo camera, the height of the point “a” may beestimated on the basis of an image distance to the point “a” and theangles α and β shown in FIG. 11.

As a different technique for height estimation, the robot 1 can move upand down the camera in the tilt direction, determine a point a₁ when atop point of the traveling object is accommodated in the screen, storesa relative angle α₁ from the camera optical axis and a camera posture β₁at that time, determine a point a₂ when a bottom point of the travelingobject is accommodated in the screen, store a relative angle α₂ from thecamera optical axis and a camera posture β₂ at that time, and estimatethe height of the traveling object by using them and a distance to theobstacle. The robot 1 may scan in the longitudinal direction by using adistance measuring sensor such as a laser range finder or a PDS(Position Sensitive Detector) sensor and estimate the height of theobject on the basis of an elevation angle of the top end of the objectand the height of the object.

The height h of the person thus estimated is compared with the thresholdDh at the step S723 shown in FIG. 10. If the height h is greater thanthe threshold Dh, then the robot 1 proceeds to step S724 and judges thatthe extracted person is not a person of caution level. In this case, therobot 1 further proceeds to step S8 shown in FIG. 2, places a limitationLv. 3 on the approach permission distance L max of the robot 1 to theperson, and sets the approach permission distance L max to L max =L 3(>L 2).

On the other hand, if the height h is equal to or less than thethreshold Dh, then the robot 1 proceeds to step S725 shown in FIG. 10and judges that the extracted person is a person of high caution level,i.e., a person who is higher in caution level than general persons. Inthis case, the robot 1 further proceeds to step S9 shown in FIG. 2, setsthe level of the approach permission distance limitation (degree ofcaution) of the robot 1 to the person to Lv. 4, and sets the approachpermission distance L_(max) to L_(max)=L₄ (>L₃).

By the way, the robot 1 uses the above-described decision techniques byswitching the decision techniques according to the distance to theobject. Specifically, if the traveling object is at a great distance andthe resolution of the distance sensor or the encoder in the camera tiltdirection is not obtained sufficiently, the person's height is estimatedwhen the traveling object gets near a place located at a distance wherethe resolution of the distance sensor or the encoder in the camera tiltdirection is obtained sufficiently. If the traveling object further getsnear and the image is not accommodated in the screen, then the robot 1moves the camera in the tilt direction, and estimates the height of theobject person by using a relative angle in the screen and a cameraposture angle. If the decision whether the traveling object is a humanbeing is difficult under the influence of the illumination condition orbackground, then the robot 1 regards every detection traveling object asa human being, and makes a decision whether to set the caution levelhigher than that for the general person.

The robot 1 has a tracking function of providing a person with anidentification code upon detecting the person and tracking the personwith time. In the case of cameras, tracking utilizing feature pointssuch as corner points or image edges is mounted. In the case of an areameasuring sensor such as a laser sensor, tracking with due regard to thetime continuity of location is mounted. If a state in which associationwith persons is difficult occurs in a confusion environment, thenidentification codes are given again and tracking is started again fromthat time point. It becomes possible to obtain time series informationof person locations by thus tracking the persons. The robot 1 also has adecision technique of measuring a traveling operation frequency, amaximum traveling velocity and a maximum traveling acceleration of aperson in the traveling plane and if these values are equal to orgreater than the set thresholds, setting the level of degree of cautionfor the person higher than that for the general person.

The above-described decision technique is used mainly for the purpose ofseparating and extracting children from general persons. The robot 1changes action restrictions according to a kind of an obstaclerecognized on the basis of the decision procedure. Here, the cautionlevels are set so as to strengthen the restrictions in the detectedobstacle order of a stationary obstacle, a traveling obstacle other thana human being, a traveling obstacle other than a child, and a child.They are arranged on the basis of large-small relations of a travelingvelocity change value and a traveling velocity maximum change rate ofthe detected obstacle. If the detected obstacle is a stationary object,then the robot 1 can avoid it at a pace of itself and consequently thecaution level becomes the lowest.

If the detected obstacle is a traveling obstacle such as a shoppingcart, then there is a restriction in its traveling locus due to theconstraint on the degree of freedom of wheels and a traveling predictionmodel having a high precision can be established. Although the level ofthe degree of caution needs to be made higher than that of a stationaryobject, therefore, the degree of caution can be set low among travelingobjects.

On the other hand, from a kinematical viewpoint, there is a possibilitythat a human being will take a posture state in which the human beingcan travel in an arbitrary direction at arbitrary timing. As comparedwith a cart or the like, traveling prediction of the human being isdifficult and the model precision also becomes low. Therefore, it isnecessary to make the caution level for the human being high. Amonghuman beings, prediction of the movement of a small child is difficultbecause the child starts to move in an unexpected direction suddenly.Therefore, the caution level of the child needs to be made high amonghuman beings.

The robot 1 in the present embodiment compares a distance which can beensured between the robot 1 and the obstacle with preset values (=0, L1,L2, L3 and L4) of the approach permission distance, and determines thetraveling velocity of the robot 1 on the basis of a result of thecomparison and preset levels of the approach permission distancelimitation, i.e., levels of the degree of caution (=no limitations, Lv.1, Lv. 2, Lv. 3 and Lv. 4). This traveling velocity is based on atraveling restriction law having two-dimensional matrix conditionstipulations in which the distance to the obstacle is increased or themaximum traveling velocity is decreased as the caution level becomeshigher as shown in FIG. 12. As a result, it becomes possible for therobot 1 to travel safely and efficiently even in a narrow place or aplace crowded with persons.

On the other hand, in the conventional robot control, an action is takenwith priority given to safety by always conducting handling in the worstcase, i.e., in the supposed example, conducting handling in the same wayas the case where a traveling object having a higher caution level isdetected, when the robot 1 has detected some traveling object. As amatter of fact, however, the robot always detects something and stops ifthe robot attempts to travel in the store according to such an actionlaw, resulting in poor efficiency.

According to the present embodiment, it is possible to provide a robotwhich conducts safe operation and has a high business capability in thestore, and its control method as heretofore described.

In the present embodiment, a decision is made by using the height of thetraveling object in order to detect a traveling object having a highercaution level. Alternatively, a different decision technique can also beused. For example, if a person detected by the robot 1 is entirelyaccommodated in a camera frame, the robot 1 may conduct circular shapematching with a contour line extracted from the traveling object, judgea region part which exists in an upper part of the traveling object andwhich is partitioned so as to have the highest correlation value to be ahead of the person, and estimate the head and body on the basis of animage size ratio between the detected person's head and whole body.

If a person's face can be caught with a sufficient size in the imagepicked up by the cameras, the robot 1 can detect three-dimensionallocation relations of face parts such as the eyes, nose and mouth on thebasis of the direction of the face and image features. The robot 1 findscorrelation values with location relations of face parts obtained fromaverage faces of respective age layers. Each of the average faces isobtained by standardizing and averaging face images of a plurality ofpersons in the same age layer in image processing by means of sizes. Ifan age layer having the highest correlation value is equal to or lessthan some threshold, here an average face of ten-year old children, therobot 1 also has a decision technique for setting the person higher incaution level than the general persons. If it is difficult to detect theface direction, it is also possible to make a decision on the basis oftwo-dimensional arrangement relations restricted to the front direction.

Additional advantages and modifications will readily occur to thoseskilled in the art. Therefore, the invention in its broader aspects isnot limited to the specific details and representative embodiments shownand described herein. Accordingly, various modifications may be madewithout departing from the spirit or scope of the general inventiveconcepts as defined by the appended claims and their equivalents.

1. A robot, comprising: a main body; a traveling mechanism configured tobe attached to the main body and to move the main body; an image pickupdevice configured to pick up an image of an object in a space includinga traveling plane; an object detector configured to detect the object inthe space based on the image picked up by the image pickup device; adistance calculator configured to calculate a distance to the objectdetected by the object detector; an obstacle decision part configured tomake a decision whether the object detected by the object detector is anobstacle based on the distance to the object calculated by the distancecalculator; a traveling object decision part configured to make adecision whether the obstacle is a traveling object based on the imagepicked up by the image pickup device and the distance calculated by thedistance calculator, and to make a decision, if the obstacle is judgedto be a traveling object, whether the traveling object is a person basedon the image picked up by the image pickup device; a caution requiringperson decision part configured to make a decision whether the travelingobject is a person against whom caution should be taken, based on theimage picked up by the image pickup device and the distance calculatedby the distance calculator; an approach permissible distance settingpart configured to set an approach permissible distance from the objectdetected by the object detector based on the image picked up by theimage pickup device and the distance calculated by the distancecalculator, the approach permissible distance setting part beingconfigured to place no restrictions on the approach permissible distancefrom the object when the object detected by the object detector is notan obstacle, to set the approach permissible distance to a firstdistance when the object detected by the object detector is an obstacleand is not a traveling object, to set the approach permissible distanceto a second distance greater than the first distance when the objectdetected by the object detector is a traveling object and is not aperson, to set the approach permissible distance to a third distancegreater than the second distance when the object detected by the objectdetector is a person and is not a person against whom caution should betaken, and to set the approach permissible distance to a fourth distancegreater than the third distance when the object detected by the objectdetector is a person against whom caution should be taken, the approachpermissible distance setting part being further configured to set adegree of caution with respect to the object, wherein the approachpermissible distance setting part places no restrictions on the degreeof caution when the object detected by the object detector is not anobstacle, sets the degree of caution to a first level when the objectdetected by the object detector is an obstacle and is not a travelingobject, to set the degree of caution to a second level when the objectdetected by the object detector is a traveling object and is not aperson, sets the degree of caution to a third level when the objectdetected by the object detector is a person and is not a person againstwhom caution should be taken, and sets the degree of caution to a fourthlevel when the object detected by the object detector is a personagainst whom caution should be taken; a velocity setting part configuredto set a velocity of the main body based on the approach permissibledistance and the degree of caution set by the approach permissibledistance setting part; and a controller configured to control thetraveling mechanism so as to cause the traveling velocity of the mainbody to become the velocity set by the velocity setting part.
 2. Acontrol method for a robot including a main body, a traveling mechanismconfigured to be attached to the main body and to move the main body, animage pickup device configured to pick up an image of an object in aspace including a traveling plane, an object detector configured todetect the object in the space based on the image picked up by the imagepickup device, and a distance calculator configured to calculate adistance to the object detected by the object detector, the controlmethod comprising: making a decision whether the obstacle is a travelingobject based on the image picked up by the image pickup device and thedistance calculated by the distance calculator, and to make a decision,if the obstacle is judged to be a traveling object, whether thetraveling object is a person based on the image picked up by the imagepickup device; making a decision whether the traveling object is aperson against whom caution should be taken, based on the image pickedup by the image pickup device and the distance calculated by thedistance calculator; setting an approach permissible distance from theobject detected by the object detector based on the image picked up bythe image pickup device and the distance calculated by the distancecalculator, the setting of the approach permissible distance comprising:placing no restrictions on the approach permissible distance from theobject when the object detected by the object detector is not anobstacle, setting the approach permissible distance to a first distancewhen the object detected by the object detector is an obstacle and isnot a traveling object, setting the approach permissible distance to asecond distance greater than the first distance when the object detectedby the object detector is a traveling object and is not a person,setting the approach permissible distance to a third distance greaterthan the second distance when the object detected by the object detectoris a person and is not a person against whom caution should be taken,and setting the approach permissible distance to a fourth distancegreater than the third distance when the object detected by the objectdetector is a person against whom caution should be taken; setting adegree of caution with respect to the object, the setting of the degreeof caution comprising: placing no restrictions on the degree of cautionwhen the object detected by the object detector is not an obstacle;setting the degree of caution to a first level when the object detectedby the object detector is an obstacle and is not a traveling object;setting the degree of caution to a second level when the object detectedby the object detector is a traveling object and is not a person;setting the degree of caution to a third level when the object detectedby the object detector is a person and is not a person against whomcaution should be taken; and setting the degree of caution to a fourthlevel when the object detected by the object detector is a personagainst whom caution should be taken; setting a velocity of the mainbody based on the approach permissible distance set by the setting ofthe approach permissible distance and the degree of caution set by thesetting of the degree of caution; and controlling the travelingmechanism so as to cause the traveling velocity of the main body tobecome the velocity set by setting of the velocity of the main body. 3.A robot, comprising: a main body; a driving mechanism attached to themain body and configured to move the main body; and a camera, whereinthe robot is configured to: detect an object based on an image picked upby the camera; calculate a distance to the detected object; determinewhether the detected object is an obstacle based on the calculateddistance; determine whether the obstacle is a traveling object based onthe image picked up by the camera and the calculated distance, and, ifthe obstacle is determined to be a traveling object, determine whetherthe traveling object is a person based on the image picked up by thecamera; determine whether the traveling object is a person against whomcaution should be taken, based on the image picked up by the camera andthe calculated distance; set an approach permissible distance from thedetected object based on the image picked up by the camera and thecalculated distance, wherein no restrictions are placed on the approachpermissible distance from the object when the object is determined notto be an obstacle; the approach permissible distance is set to a firstdistance when the object is determined to be an obstacle, but not atraveling object; the approach permissible distance is set to a seconddistance greater than the first distance when the object is determinedto be a traveling object, but not a person; the approach permissibledistance is set to a third distance greater than the second distancewhen the object is determined to be a person, but not a person againstwhom caution should be taken; and the approach permissible distance isset to a fourth distance greater than the third distance when the objectis determined to be a person against whom caution should be taken; set adegree of caution with respect to the object, wherein no restrictionsare placed on the degree of caution when the object detected by theobject detector is not an obstacle, the degree of caution is set to afirst level when the object detected by the object detector is anobstacle and is not a traveling object, the degree of caution is set asecond level when the object detected by the object detector is atraveling object and is not a person, the degree of caution is set to athird level when the object detected by the object detector is a personand is not a person against whom caution should be taken, and the degreeof caution is set to a fourth level when the object detected by theobject detector is a person against whom caution should be taken; andset a velocity of the main body based on the approach permissibledistance and the degree of caution.